---------------------------------------------------------------------------------------------------------------------------------
      name:  <unnamed>
       log:  /Users/simonvaraine/Documents/Doctorat/Ecriture d'articles/Double check or dont check/Submission JEPS/Final version/
> Manipulation_checks_replication_log.log
  log type:  text
 opened on:   6 Sep 2022, 10:41:08

. do "/Users/simonvaraine/Documents/Doctorat/Ecriture d'articles/Double check or dont check/Submission JEPS/Final version/Replica
> tion files/Manipulation_checks_replication_code.do"

. 
. 
. clear

. 
. * * * * First download the replication dataset "Manipulation_checks_replication_data.dta" from the JEPS Dataverse site
. 
. * * * Use the path to the folder in which you uploaded the data as directory
. 
. *cd "/THE PATH TO THE FOLDER/"
. cd "/Users/simonvaraine/Documents/Doctorat/Ecriture d'articles/Double check or dont check/Submission JEPS/Final version/Replica
> tion files"
/Users/simonvaraine/Documents/Doctorat/Ecriture d'articles/Double check or dont check/Submission JEPS/Final version/Replication f
> iles

. 
. * * * Open the data
. 
. use Manipulation_checks_replication_data

. 
. 
. 
. 
. * * * * * Compute the results of the paper 
. 
. * * * * Main manuscript
. 
. * * * Proportion of subjects who failed/succeded the manipulation check
. 
. tab Experimental_treatment Response_manipulation_check, cell

+-----------------+
| Key             |
|-----------------|
|    frequency    |
| cell percentage |
+-----------------+

Experiment |
        al |
treatment. |
       0 = |   Response to the manipulation
Prosperity | check question. 0 = Prosperity,
     , 1 = |       1 = Stability, 2 = 
  Decline. |         0          1          2 |     Total
-----------+---------------------------------+----------
         0 |     1,081        571        321 |     1,973 
           |     27.58      14.57       8.19 |     50.33 
-----------+---------------------------------+----------
         1 |        89        481      1,377 |     1,947 
           |      2.27      12.27      35.13 |     49.67 
-----------+---------------------------------+----------
     Total |     1,170      1,052      1,698 |     3,920 
           |     29.85      26.84      43.32 |    100.00 


. * Interpretation: 27.58% + 35.13% of subjects succeeded 
. 
. * * * Figure 1: Share of subjects by experimental treatment and by response to the manipulation check question (with 95% Confid
> ence Interval)
. 
. * * Proportion of subjects in each treatment (with 95% Confidence Interval)
. 
. proportion  Experimental_treatment

Proportion estimation               Number of obs    =    3949

------------------------------------------------------------------------
                       | Proportion   Std. Err.     [95% Conf. Interval]
-----------------------+------------------------------------------------
Experimental_treatment |
                     0 |   .5018992   .0079575      .4863012    .5174936
                     1 |   .4981008   .0079575      .4825064    .5136988
------------------------------------------------------------------------

. 
. * * Proportion of subjects by response to the manipulation check (with 95% Confidence Interval)
. 
. proportion  Response_manipulation_check

Proportion estimation               Number of obs    =    3920

-----------------------------------------------------------------------------
                            | Proportion   Std. Err.     [95% Conf. Interval]
----------------------------+------------------------------------------------
Response_manipulation_check |
                          0 |   .2984694   .0073095      .2843391    .3129948
                          1 |   .2683673   .0070782      .2547191    .2824697
                          2 |   .4331633   .0079153      .4177152    .4487424
-----------------------------------------------------------------------------

. 
. * * To create the figure, copy the statistics from the previous commands, here:            (no easy to use graph command to plo
> t proportions and CIs to my knowledge)
. 
. * For the proportion of subjects in each treatment (with 95% Confidence Interval): 
. 
. clear

. input float(n Experimental_treatment prop se lower upper)

             n  Experim~t       prop         se      lower      upper
  1. 1 0 .5018992   .0079575      .4863012    .5174936
  2. 2 1  .4981008   .0079575      .4825064    .5136988
  3. end

. 
. * Plot the statistics
. 
. twoway (bar prop Experimental_treatment) (rcap lower upper Experimental_treatment) , ysc(r(0 .5)) name(prop1, replace)

. 
. * For the proportion of subjects in each treatment (with 95% Confidence Interval): 
. 
. clear

. input float(n EXP2_Eco prop se lower upper)

             n   EXP2_Eco       prop         se      lower      upper
  1. 1 0   .2984694   .0073095      .2843391    .3129948
  2. 2 1   .2683673   .0070782      .2547191    .2824697
  3. 3 2   .4331633   .0079153      .4177152    .4487424
  4. end

. 
. * Plot the statistics
. 
. twoway (bar prop EXP2_Eco) (rcap lower upper EXP2_Eco) , ysc(r(0 .5)) name(prop2, replace)

. 
. * Combine the two plots
. 
. graph combine prop1 prop2, ycommon 

. 
. 
. * * * Back to the raw data
. clear

. use Manipulation_checks_replication_data

. 
. 
. * * * Two-way Anova: Relationship between perception of the economy and responses to the manipulation check question
. 
. anova Perception_eco Response_manipulation_check

                           Number of obs =    3731     R-squared     =  0.0291
                           Root MSE      = 2.11563     Adj R-squared =  0.0285

                  Source |  Partial SS    df       MS           F     Prob > F
             ------------+----------------------------------------------------
                   Model |  499.464152     2  249.732076      55.79     0.0000
                         |
             Response_~k |  499.464152     2  249.732076      55.79     0.0000
                         |
                Residual |  16686.1445  3728  4.47589714   
             ------------+----------------------------------------------------
                   Total |  17185.6087  3730  4.60740179   

. 
. * * * Figure 2: Share of responses to the manipulation check question depending on the perception of the national economy prior
>  to the experiment (with 95% Confidence Interval)
. 
. * * Modelize the probability of each response to the manipulation check question based on a multinomial logistic regression
. 
. mlogit Response_manipulation_check i.Perception_eco

Iteration 0:   log likelihood = -4008.2975  
Iteration 1:   log likelihood = -3937.6362  
Iteration 2:   log likelihood = -3936.2142  
Iteration 3:   log likelihood = -3936.2043  
Iteration 4:   log likelihood = -3936.2043  

Multinomial logistic regression                   Number of obs   =       3731
                                                  LR chi2(20)     =     144.19
                                                  Prob > chi2     =     0.0000
Log likelihood = -3936.2043                       Pseudo R2       =     0.0180

--------------------------------------------------------------------------------
Response_man~k |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
---------------+----------------------------------------------------------------
0              |
Perception_eco |
            1  |   .1502368   .3071709     0.49   0.625    -.4518072    .7522808
            2  |   .1417979   .2594548     0.55   0.585    -.3667242      .65032
            3  |    .321868   .2391963     1.35   0.178    -.1469481    .7906841
            4  |   .5004392   .2313881     2.16   0.031     .0469268    .9539516
            5  |   .6562891   .2264324     2.90   0.004     .2124898    1.100088
            6  |   .8195667   .2265622     3.62   0.000      .375513     1.26362
            7  |   .9140674   .2312926     3.95   0.000     .4607423    1.367393
            8  |   .9882149   .2561529     3.86   0.000     .4861645    1.490265
            9  |   .6784224   .3446027     1.97   0.049     .0030135    1.353831
           10  |   .8371534   .3694495     2.27   0.023     .1130456    1.561261
               |
         _cons |  -.9769154   .2073999    -4.71   0.000    -1.383412   -.5704191
---------------+----------------------------------------------------------------
1              |
Perception_eco |
            1  |   .8715548    .427721     2.04   0.042     .0332371    1.709873
            2  |   .6613985   .3890703     1.70   0.089    -.1011653    1.423962
            3  |    1.07487   .3615295     2.97   0.003     .3662851    1.783455
            4  |   1.539292   .3509272     4.39   0.000     .8514876    2.227097
            5  |   1.928144   .3457293     5.58   0.000     1.250527    2.605761
            6  |   1.806654   .3477696     5.19   0.000     1.125038     2.48827
            7  |   1.880743   .3513115     5.35   0.000     1.192185    2.569301
            8  |   2.128637   .3669095     5.80   0.000     1.409508    2.847767
            9  |   1.924955   .4289788     4.49   0.000     1.084172    2.765738
           10  |   2.223448   .4418631     5.03   0.000     1.357412    3.089483
               |
         _cons |  -2.140066   .3343123    -6.40   0.000    -2.795306   -1.484826
---------------+----------------------------------------------------------------
2              |  (base outcome)
--------------------------------------------------------------------------------

. 
. * * Compute the predictions and confidence intervals from each response to the manipulation check question
. 
. margins Perception_eco, predict(outcome(0)) saving(margins1, replace)

Adjusted predictions                              Number of obs   =       3731
Model VCE    : OIM

Expression   : Pr(Response_manipulation_check==0), predict(outcome(0))

--------------------------------------------------------------------------------
               |            Delta-method
               |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
---------------+----------------------------------------------------------------
Perception_eco |
            0  |   .2519685    .038524     6.54   0.000     .1764629    .3274741
            1  |   .2545455   .0415333     6.13   0.000     .1731416    .3359493
            2  |   .2610619   .0292161     8.94   0.000     .2037995    .3183244
            3  |   .2786458   .0228789    12.18   0.000      .233804    .3234876
            4  |   .2862454   .0194873    14.69   0.000     .2480509    .3244398
            5  |   .2863014   .0167305    17.11   0.000     .2535103    .3190925
            6  |   .3323398   .0181848    18.28   0.000     .2966983    .3679813
            7  |   .3464419   .0205914    16.82   0.000     .3060835    .3868004
            8  |   .3371212   .0290943    11.59   0.000     .2800974     .394145
            9  |   .2911392   .0511113     5.70   0.000     .1909628    .3913156
           10  |   .2941176   .0552551     5.32   0.000     .1858197    .4024156
--------------------------------------------------------------------------------
(note: file margins1.dta not found)

. margins Perception_eco, predict(outcome(1)) saving(margins2, replace)

Adjusted predictions                              Number of obs   =       3731
Model VCE    : OIM

Expression   : Pr(Response_manipulation_check==1), predict(outcome(1))

--------------------------------------------------------------------------------
               |            Delta-method
               |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
---------------+----------------------------------------------------------------
Perception_eco |
            0  |   .0787402   .0238994     3.29   0.001     .0318982    .1255822
            1  |   .1636364   .0352729     4.64   0.000     .0945028      .23277
            2  |   .1371681   .0228842     5.99   0.000      .092316    .1820203
            3  |   .1848958   .0198109     9.33   0.000     .1460672    .2237245
            4  |   .2527881   .0187374    13.49   0.000     .2160635    .2895127
            5  |   .3191781   .0172533    18.50   0.000     .2853623    .3529939
            6  |   .2786885   .0173085    16.10   0.000     .2447645    .3126126
            7  |   .2846442   .0195273    14.58   0.000     .2463714    .3229169
            8  |   .3295455   .0289295    11.39   0.000     .2728447    .3862462
            9  |   .3164557    .052327     6.05   0.000     .2138966    .4190148
           10  |   .3676471    .058471     6.29   0.000     .2530459    .4822482
--------------------------------------------------------------------------------
(note: file margins2.dta not found)

. margins Perception_eco, predict(outcome(2)) saving(margins3, replace)

Adjusted predictions                              Number of obs   =       3731
Model VCE    : OIM

Expression   : Pr(Response_manipulation_check==2), predict(outcome(2))

--------------------------------------------------------------------------------
               |            Delta-method
               |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
---------------+----------------------------------------------------------------
Perception_eco |
            0  |   .6692913   .0417473    16.03   0.000     .5874681    .7511146
            1  |   .5818182   .0470305    12.37   0.000       .48964    .6739963
            2  |   .6017699   .0325633    18.48   0.000     .5379471    .6655928
            3  |   .5364583   .0254476    21.08   0.000      .486582    .5863347
            4  |   .4609665   .0214907    21.45   0.000     .4188455    .5030876
            5  |   .3945205   .0180894    21.81   0.000     .3590661     .429975
            6  |   .3889717   .0188204    20.67   0.000     .3520844     .425859
            7  |   .3689139   .0208803    17.67   0.000     .3279893    .4098384
            8  |   .3333333   .0290129    11.49   0.000      .276469    .3901977
            9  |   .3924051   .0549365     7.14   0.000     .2847315    .5000786
           10  |   .3382353   .0573729     5.90   0.000     .2257865    .4506841
--------------------------------------------------------------------------------
(note: file margins3.dta not found)

. 
. * * Plot the predictions
. 
. combomarginsplot margins1 margins2 margins3

  Variables that uniquely identify margins: Perception_eco _filenumber

. 
. 
. * * * T-test: Relationship between the experimental treatment and subjects' perception of the national economy prior to the sur
> vey experiment
. 
. * * With all subjects
. 
. ttest Perception_eco, by(Experimental_treatment)

Two-sample t test with equal variances
------------------------------------------------------------------------------
   Group |     Obs        Mean    Std. Err.   Std. Dev.   [95% Conf. Interval]
---------+--------------------------------------------------------------------
       0 |    1880    5.054255    .0492503    2.135444    4.957664    5.150846
       1 |    1874     5.01174    .0499394    2.161862    4.913797    5.109682
---------+--------------------------------------------------------------------
combined |    3754    5.033031     .035066    2.148491    4.964281    5.101782
---------+--------------------------------------------------------------------
    diff |            .0425157     .070138               -.0949967    .1800281
------------------------------------------------------------------------------
    diff = mean(0) - mean(1)                                      t =   0.6062
Ho: diff = 0                                     degrees of freedom =     3752

    Ha: diff < 0                 Ha: diff != 0                 Ha: diff > 0
 Pr(T < t) = 0.7278         Pr(|T| > |t|) = 0.5444          Pr(T > t) = 0.2722

. 
. * * Excluding subjects who failed the manipulation check
. 
. ttest Perception_eco if (Experimental_treatment == 0 & Response_manipulation_check == 0) | (Experimental_treatment == 1 & Respo
> nse_manipulation_check == 2), by(Experimental_treatment)

Two-sample t test with equal variances
------------------------------------------------------------------------------
   Group |     Obs        Mean    Std. Err.   Std. Dev.   [95% Conf. Interval]
---------+--------------------------------------------------------------------
       0 |    1045     5.11866    .0641047    2.072278    4.992872    5.244449
       1 |    1323    4.716553     .059619    2.168525    4.599595    4.833511
---------+--------------------------------------------------------------------
combined |    2368    4.894003    .0438843    2.135502    4.807948    4.980059
---------+--------------------------------------------------------------------
    diff |             .402107     .088011                .2295202    .5746938
------------------------------------------------------------------------------
    diff = mean(0) - mean(1)                                      t =   4.5688
Ho: diff = 0                                     degrees of freedom =     2366

    Ha: diff < 0                 Ha: diff != 0                 Ha: diff > 0
 Pr(T < t) = 1.0000         Pr(|T| > |t|) = 0.0000          Pr(T > t) = 0.0000

. 
. * * * Figure 3: Average perception of the national economy prior to the experiment depending on the experimental treatment (wit
> h 95% Confidence Interval)
. 
. * * Modelize the perception of the national economy depending on the experimental treatment for all subjects based on a linear 
> regression
. 
. reg Perception_eco i.Experimental_treatment 

      Source |       SS       df       MS              Number of obs =    3754
-------------+------------------------------           F(  1,  3752) =    0.37
       Model |  1.69641592     1  1.69641592           Prob > F      =  0.5444
    Residual |  17322.2077  3752  4.61679309           R-squared     =  0.0001
-------------+------------------------------           Adj R-squared = -0.0002
       Total |  17323.9041  3753  4.61601495           Root MSE      =  2.1487

------------------------------------------------------------------------------------------
          Perception_eco |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------------------+----------------------------------------------------------------
1.Experimental_treatment |  -.0425157    .070138    -0.61   0.544    -.1800281    .0949967
                   _cons |   5.054255   .0495554   101.99   0.000     4.957097    5.151414
------------------------------------------------------------------------------------------

. margins Experimental_treatment

Adjusted predictions                              Number of obs   =       3754
Model VCE    : OLS

Expression   : Linear prediction, predict()

----------------------------------------------------------------------------------------
                       |            Delta-method
                       |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
-----------------------+----------------------------------------------------------------
Experimental_treatment |
                    0  |   5.054255   .0495554   101.99   0.000     4.957097    5.151414
                    1  |    5.01174   .0496347   100.97   0.000     4.914426    5.109053
----------------------------------------------------------------------------------------

. marginsplot, ysc(r(4.5 5.5)) name(marginsa, replace)

  Variables that uniquely identify margins: Experimental_treatment

. 
. * * Modelize the perception of the national economy depending on the experimental treatment excluding subjects who failed the m
> anipulation check based on a linear regression
. 
. reg Perception_eco i.Experimental_treatment if (Experimental_treatment == 0 & Response_manipulation_check == 0) | (Experimental
> _treatment == 1 & Response_manipulation_check == 2)

      Source |       SS       df       MS              Number of obs =    2368
-------------+------------------------------           F(  1,  2366) =   20.87
       Model |  94.4012406     1  94.4012406           Prob > F      =  0.0000
    Residual |  10699.9936  2366  4.52239797           R-squared     =  0.0087
-------------+------------------------------           Adj R-squared =  0.0083
       Total |  10794.3948  2367   4.5603696           Root MSE      =  2.1266

------------------------------------------------------------------------------------------
          Perception_eco |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------------------+----------------------------------------------------------------
1.Experimental_treatment |   -.402107    .088011    -4.57   0.000    -.5746938   -.2295202
                   _cons |    5.11866   .0657849    77.81   0.000     4.989658    5.247662
------------------------------------------------------------------------------------------

. margins Experimental_treatment

Adjusted predictions                              Number of obs   =       2368
Model VCE    : OLS

Expression   : Linear prediction, predict()

----------------------------------------------------------------------------------------
                       |            Delta-method
                       |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
-----------------------+----------------------------------------------------------------
Experimental_treatment |
                    0  |    5.11866   .0657849    77.81   0.000     4.989658    5.247662
                    1  |   4.716553   .0584661    80.67   0.000     4.601903    4.831203
----------------------------------------------------------------------------------------

. marginsplot, ysc(r(4.5 5.5)) name(marginsb, replace)

  Variables that uniquely identify margins: Experimental_treatment

. 
. * * Combine the two plots
. 
. graph combine marginsa marginsb, nocopies ycommon

. 
. 
. * * * Table 1: Results from linear regression models of the level of nostalgia
. 
. * * Model 1
. 
. reg Nostalgia i.Experimental_treatment if (Experimental_treatment == 0 & Response_manipulation_check == 0) | (Experimental_trea
> tment == 1 & Response_manipulation_check == 2)

      Source |       SS       df       MS              Number of obs =    2395
-------------+------------------------------           F(  1,  2393) =   14.38
       Model |  15.9338397     1  15.9338397           Prob > F      =  0.0002
    Residual |  2650.72002  2393  1.10769746           R-squared     =  0.0060
-------------+------------------------------           Adj R-squared =  0.0056
       Total |  2666.65386  2394   1.1138905           Root MSE      =  1.0525

------------------------------------------------------------------------------------------
               Nostalgia |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------------------+----------------------------------------------------------------
1.Experimental_treatment |   .1641858   .0432898     3.79   0.000     .0792964    .2490753
                   _cons |   3.343691    .032296   103.53   0.000      3.28036    3.407022
------------------------------------------------------------------------------------------

. 
. * * Model 2
. 
. reg Nostalgia i.Experimental_treatment Perception_eco if (Experimental_treatment == 0 & Response_manipulation_check == 0) | (Ex
> perimental_treatment == 1 & Response_manipulation_check == 2)

      Source |       SS       df       MS              Number of obs =    2311
-------------+------------------------------           F(  2,  2308) =  105.99
       Model |  216.793969     2  108.396985           Prob > F      =  0.0000
    Residual |  2360.41417  2308  1.02270978           R-squared     =  0.0841
-------------+------------------------------           Adj R-squared =  0.0833
       Total |  2577.20814  2310  1.11567452           Root MSE      =  1.0113

------------------------------------------------------------------------------------------
               Nostalgia |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------------------+----------------------------------------------------------------
1.Experimental_treatment |    .114489    .042511     2.69   0.007     .0311253    .1978528
          Perception_eco |  -.1384393   .0098879   -14.00   0.000    -.1578294   -.1190492
                   _cons |   4.054361   .0596611    67.96   0.000     3.937366    4.171356
------------------------------------------------------------------------------------------

. 
. * * Model 3
. 
. reg Nostalgia i.Experimental_treatment 

      Source |       SS       df       MS              Number of obs =    3797
-------------+------------------------------           F(  1,  3795) =    3.20
       Model |  3.44740571     1  3.44740571           Prob > F      =  0.0735
    Residual |  4082.23603  3795  1.07568802           R-squared     =  0.0008
-------------+------------------------------           Adj R-squared =  0.0006
       Total |  4085.68343  3796  1.07631281           Root MSE      =  1.0372

------------------------------------------------------------------------------------------
               Nostalgia |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------------------+----------------------------------------------------------------
1.Experimental_treatment |   .0602683   .0336656     1.79   0.074     -.005736    .1262727
                   _cons |   3.425598   .0236574   144.80   0.000     3.379216    3.471981
------------------------------------------------------------------------------------------

. 
. 
. 
. 
. * * * * Online appendix
. 
. * * * Table 1: Results from linear regression models of the level of nostalgia
. 
. * * Model 1
. 
. reg Nostalgia i.Experimental_treatment i.Country_num if (Experimental_treatment == 0 & Response_manipulation_check == 0) | (Exp
> erimental_treatment == 1 & Response_manipulation_check == 2)

      Source |       SS       df       MS              Number of obs =    2395
-------------+------------------------------           F(  6,  2388) =   14.54
       Model |  93.9844742     6   15.664079           Prob > F      =  0.0000
    Residual |  2572.66939  2388  1.07733224           R-squared     =  0.0352
-------------+------------------------------           Adj R-squared =  0.0328
       Total |  2666.65386  2394   1.1138905           Root MSE      =  1.0379

------------------------------------------------------------------------------------------
               Nostalgia |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------------------+----------------------------------------------------------------
1.Experimental_treatment |   .1313821   .0428895     3.06   0.002     .0472777    .2154866
                         |
             Country_num |
                 France  |   .1709678   .0773284     2.21   0.027       .01933    .3226056
                Germany  |   .1235995   .0770415     1.60   0.109    -.0274756    .2746746
                  Italy  |   .4224353   .0797198     5.30   0.000     .2661081    .5787625
            Netherlands  |  -.1758673   .0770273    -2.28   0.023    -.3269146   -.0248199
                  Spain  |   .1651675   .0778581     2.12   0.034      .012491    .3178439
                         |
                   _cons |   3.246563   .0631374    51.42   0.000     3.122754    3.370373
------------------------------------------------------------------------------------------

. 
. * * Model 2
. 
. reg Nostalgia i.Experimental_treatment Perception_eco i.Country_num if (Experimental_treatment == 0 & Response_manipulation_che
> ck == 0) | (Experimental_treatment == 1 & Response_manipulation_check == 2)

      Source |       SS       df       MS              Number of obs =    2311
-------------+------------------------------           F(  7,  2303) =   35.79
       Model |  252.879134     7  36.1255906           Prob > F      =  0.0000
    Residual |    2324.329  2303   1.0092614           R-squared     =  0.0981
-------------+------------------------------           Adj R-squared =  0.0954
       Total |  2577.20814  2310  1.11567452           Root MSE      =  1.0046

------------------------------------------------------------------------------------------
               Nostalgia |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------------------+----------------------------------------------------------------
1.Experimental_treatment |   .0989661   .0423469     2.34   0.020     .0159241    .1820081
          Perception_eco |  -.1324637   .0105966   -12.50   0.000    -.1532435   -.1116839
                         |
             Country_num |
                 France  |  -.0850783   .0800433    -1.06   0.288    -.2420428    .0718862
                Germany  |   .0788003   .0766526     1.03   0.304     -.071515    .2291156
                  Italy  |   .2194672   .0813717     2.70   0.007     .0598977    .3790366
            Netherlands  |  -.1801508   .0766402    -2.35   0.019    -.3304419   -.0298598
                  Spain  |   .0290815   .0778296     0.37   0.709     -.123542     .181705
                         |
                   _cons |   4.028125   .0891909    45.16   0.000     3.853222    4.203028
------------------------------------------------------------------------------------------

. 
. * * Model 3
. 
. reg Nostalgia i.Experimental_treatment i.Country_num 

      Source |       SS       df       MS              Number of obs =    3797
-------------+------------------------------           F(  6,  3790) =   16.95
       Model |  106.754267     6  17.7923778           Prob > F      =  0.0000
    Residual |  3978.92917  3790  1.04984938           R-squared     =  0.0261
-------------+------------------------------           Adj R-squared =  0.0246
       Total |  4085.68343  3796  1.07631281           Root MSE      =  1.0246

------------------------------------------------------------------------------------------
               Nostalgia |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------------------+----------------------------------------------------------------
1.Experimental_treatment |   .0540506   .0332899     1.62   0.105    -.0112173    .1193185
                         |
             Country_num |
                 France  |   .1289375   .0572074     2.25   0.024     .0167772    .2410978
                Germany  |   .0751714   .0569779     1.32   0.187     -.036539    .1868818
                  Italy  |   .3890225   .0565361     6.88   0.000     .2781784    .4998666
            Netherlands  |   -.152471   .0569004    -2.68   0.007    -.2640294   -.0409126
                  Spain  |   .1139357    .058599     1.94   0.052    -.0009529    .2288243
                         |
                   _cons |   3.335903    .043668    76.39   0.000     3.250288    3.421518
------------------------------------------------------------------------------------------

. 
. 
. * * * Figure 1: Share of subjects who failed the manipulation check depending on the time they spent on the survey (with 95% Co
> nfidence Interval)
. 
. * * Extract subjects' decile of survey duration based on the total survey duraction in seconds
. 
. egen Deciles_survey_duraction =xtile( Survey_duration ), nq(10)

. 
. * * Generate a variable capturing Failure to the manipulation check
. 
. gen Failure_manipulation_check = 1

. replace Failure_manipulation_check = 0 if (Experimental_treatment == 0 & Response_manipulation_check == 0) | (Experimental_trea
> tment == 1 & Response_manipulation_check == 2)
(2458 real changes made)

. 
. * * Modelize the proportion of subjects who failed the manipulation check depending on the time they spent on the survey based 
> on a logistic regression
. 
. logit Failure_manipulation_check i.Deciles_survey_duraction

Iteration 0:   log likelihood =   -2617.63  
Iteration 1:   log likelihood =   -2507.89  
Iteration 2:   log likelihood = -2507.8132  
Iteration 3:   log likelihood = -2507.8132  

Logistic regression                               Number of obs   =       3949
                                                  LR chi2(9)      =     219.63
                                                  Prob > chi2     =     0.0000
Log likelihood = -2507.8132                       Pseudo R2       =     0.0420

--------------------------------------------------------------------------------------------
Failure_manipulation_check |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
---------------------------+----------------------------------------------------------------
  Deciles_survey_duraction |
                        2  |  -1.087868   .1494461    -7.28   0.000    -1.380777   -.7949594
                        3  |  -1.448506   .1516115    -9.55   0.000    -1.745659   -1.151353
                        4  |  -1.779036   .1563641   -11.38   0.000    -2.085504   -1.472568
                        5  |  -1.642023   .1542975   -10.64   0.000     -1.94444   -1.339605
                        6  |  -1.549016   .1527643   -10.14   0.000    -1.848429   -1.249604
                        7  |  -1.529968   .1530016   -10.00   0.000    -1.829846    -1.23009
                        8  |  -1.696982   .1549228   -10.95   0.000    -2.000625   -1.393339
                        9  |  -1.481183   .1522429    -9.73   0.000    -1.779574   -1.182793
                       10  |  -1.414697    .151541    -9.34   0.000    -1.711711   -1.117682
                           |
                     _cons |   .8520883   .1094879     7.78   0.000      .637496    1.066681
--------------------------------------------------------------------------------------------

. margins Deciles_survey_duraction

Adjusted predictions                              Number of obs   =       3949
Model VCE    : OIM

Expression   : Pr(Failure_manipulation_check), predict()

------------------------------------------------------------------------------------------
                         |            Delta-method
                         |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------------------+----------------------------------------------------------------
Deciles_survey_duraction |
                      1  |    .701005   .0229483    30.55   0.000     .6560271    .7459829
                      2  |   .4413265   .0250793    17.60   0.000     .3921719    .4904811
                      3  |   .3551637   .0240184    14.79   0.000     .3080886    .4022389
                      4  |   .2835443   .0226781    12.50   0.000     .2390961    .3279926
                      5  |   .3121827   .0233449    13.37   0.000     .2664275     .357938
                      6  |   .3324937   .0236442    14.06   0.000      .286152    .3788354
                      7  |   .3367347   .0238696    14.11   0.000     .2899512    .3835182
                      8  |   .3005051   .0230394    13.04   0.000     .2553487    .3456614
                      9  |   .3477157   .0239929    14.49   0.000     .3006905    .3947409
                     10  |   .3629442   .0242248    14.98   0.000     .3154644    .4104239
------------------------------------------------------------------------------------------

. 
. * * Plot the predictions
. 
. marginsplot

  Variables that uniquely identify margins: Deciles_survey_duraction

. 
. 
. 
. 
. 
. 
. 
end of do-file

. log close
      name:  <unnamed>
       log:  /Users/simonvaraine/Documents/Doctorat/Ecriture d'articles/Double check or dont check/Submission JEPS/Final version/
> Manipulation_checks_replication_log.log
  log type:  text
 closed on:   6 Sep 2022, 10:42:00
---------------------------------------------------------------------------------------------------------------------------------
